Detection of near-field camera obstruction

ABSTRACT

A method ( 100 ) is provided for detecting an obstruction within a field of view of a camera ( 12 ) from an image ( 200 ) captured by the camera ( 12 ). The method ( 100 ) includes: analyzing the image ( 200 ) by applying edge detection ( 104 ) to the image ( 200 ), identifying ( 108 ) regions of the image ( 200 ) lacking edge content and comparing ( 112 ) a size of the identified regions to a threshold; and determining if there is an obstruction based upon a result of said comparison.

BACKGROUND

The present inventive subject matter relates generally to the art ofautomated cameras. Particular but not exclusive relevance is found inconnection with red light and/or other traffic cameras. Accordingly, thepresent specification makes specific reference thereto. It is to beappreciated however that aspects of the present inventive subject matterare also equally amenable to other like applications.

To capture high quality images with red light, traffic and/or other likeautomated and/or unattended cameras it is commonly desirable to have anunobstructed field of view (FoV) in which objects of interest may belocated. Should the FoV be obstructed, objects of interest, e.g., suchas vehicles, drivers and/or license plates, may not be accuratelyvisualized and/or identifiable in images captured by the camera. Forexample, accurate visualization and/or identification of such objects incaptured images are often important for law enforcement purposes and/orthe issuing of traffic citation.

Over time, a camera's FoV may become obstructed by an object in the FoVnear the camera. For example, while not initially present, obstructionsnear the camera may appear due to the growth of plant foliage, icebuild-up on the camera lens or porthole, graffiti or debris on thecamera lens or porthole, etc. Such obstructions can sufficiently blockor obscure various regions sought to be captured in an image obtained bythe camera. In turn, one or more objects of interest otherwise sought tobe captured in such an image may not be sufficiently visualized and/orreadily identifiable in the image. Accordingly, law enforcement or otheractions reliant on accurate visualization and/or identification of oneor more target objects in a captured image may be frustrated. Moreover,some more advance camera systems may be triggered to capture an image inresponse to events occurring in a scene observed by the camera, e.g.,such as the detection of a vehicle or vehicle movement within the scene.Where such an event is obscured from view by an obstruction, the cameramay not capture an otherwise desired image because the event was notdetected.

Traditionally, operators of automated/unattended cameras such as thosementioned above relied on human labor-intensive practices to monitor,check and/or verify obstruction-free operation. For example, an operatormay periodically or intermittently conduct a manual review of imagesobtained from a camera and visually inspect them for obstructions. Suchan operator may commonly be assigned a significant number of cameras tocheck on a fairly frequent basis. Accordingly, such a process can berepetitive and prone to human oversight and/or error. Additionally, amaintenance technician may be assigned to manually inspect camerainstallations in the field at periodic or intermittent intervals. Again,this is a labor-intensive process prone to human oversight and/or error.

Alternately, automated methods have been developed to detect cameraobstructions from an obtained test image. For example, one such methodemploys a reference image obtained from an unobstructed camera. In thiscase, the reference image and test image are subtracted from one anotherto detect variations therebetween, wherein a detected variation is thendeemed indicative of an obstruction in the test image. Such subtractivemethods, however, can have certain limitations and/or drawbacks. Forexample, in dynamically changing scenes, e.g., such as a trafficintersection, objects and/or object locations within the scene may varyfrom image to image. For example, different vehicles or pedestrians mayappear in different images or appear at different locations withindifferent images. Accordingly, by image subtraction from a referencewhich may not include the same dynamically changing elements, theresulting variations may falsely indicate an obstruction. Additionally,a change in the camera alignment and/or imaging conditions (e.g., suchas illumination level) may produce variations in the subtracted imagewhich can again lead to a false indication of an obstruction.Accordingly, such subtraction methods commonly have to account fordynamically varying scenes in order to accurately detect obstructions.The image subtraction and/or aforementioned accounting for dynamicallyvarying scenes can be time intensive and may put further demands and/orcomplexity on a processor executing the same. Therefore, it hasheretofore remained desirable to have an obstruction detection methodthat is not dependent upon a reference image in this manner.

Accordingly, a new and/or improved method, system and/or apparatus formonitoring, detecting and/or reporting obstructions in a camera's FoV isdisclosed which addresses the above-referenced problem(s) and/or others.

SUMMARY

This summary is provided to introduce concepts related to the presentinventive subject matter. This summary is not intended to identifyessential features of the claimed subject matter nor is it intended foruse in determining or limiting the scope of the claimed subject matter.

In accordance with one embodiment, a method is provided for detecting anobstruction within a field of view of a camera from an image captured bythe camera. The method includes: analyzing the image by applying edgedetection to the image, identifying regions of the image lacking edgecontent and comparing a size of the identified regions to a threshold;and determining if there is an obstruction based upon a result of saidcomparison.

In accordance with another embodiment, a method is again provided fordetecting an obstruction within a field of view of a camera from animage captured by the camera. The method includes: analyzing the imagewith a computer processor to identify regions of the image which are outof focus; comparing a size of the identified regions to a threshold; anddetermining if there is an obstruction based upon a result of saidcomparison.

In accordance with still another embodiment, a camera system includes: acamera that obtains an image; and an image processor that analyzes theimage to determine if there is an obstruction in the camera's field ofview. Suitably, the analyzing includes: applying edge detection to theimage; identifying regions of the image lacking edge content; comparinga size of the identified regions to a threshold; and determining ifthere is an obstruction based upon a result of said comparison.

Numerous advantages and benefits of the inventive subject matterdisclosed herein will become apparent to those of ordinary skill in theart upon reading and understanding the present specification.

BRIEF DESCRIPTION OF THE DRAWING(S)

The following detailed description makes reference to the figures in theaccompanying drawings. However, the inventive subject matter disclosedherein may take form in various components and arrangements ofcomponents, and in various steps and arrangements of steps. The drawingsare only for purposes of illustrating exemplary and/or preferredembodiments and are not to be construed as limiting. Further, it is tobe appreciated that the drawings may not be to scale.

FIG. 1 is a diagrammatic illustration showing an exemplary camera systemsuitable for practicing aspect of the present inventive subject matter.

FIG. 2 is a flow chart illustrating an exemplary process for analyzingan image in accordance with aspects of the present inventive subjectmatter.

FIG. 3 is an illustration showing an exemplary image suitable foranalysis in accordance with aspect of the present inventive subjectmatter.

FIG. 4 is an illustration showing a resulting edge and/or gradient mapgenerated from the image shown in FIG. 3 as produced in accordance withaspects of the present inventive subject matter.

DETAILED DESCRIPTION OF THE EMBODIMENT(S)

For clarity and simplicity, the present specification shall refer tostructural and/or functional elements, relevant standards and/orprotocols, and other components that are commonly known in the artwithout further detailed explanation as to their configuration oroperation except to the extent they have been modified or altered inaccordance with and/or to accommodate the preferred embodiment(s)presented herein.

Generally, the present specification describes a method, process,apparatus and/or system for detecting a near-field obstruction within acamera's FoV that is reference-image independent, i.e., it does notutilize a reference image for the purpose of such detection. Inpractice, the described method, process, apparatus and/or system employsedge detection over an image captured by the camera and computes orotherwise determines or measures a local edge density of the image withan appropriately sized window. Suitably, a region of interest of animage scene is largely in focus under nominal (i.e., not obstructed)conditions. Consequently, the features are sharply captured and exhibitcertain homogeneity in edge density or more generally regions with highlocal gradients. Conversely, an obstruction that is sufficientlynear-field compared to the focusing range is generally not in focus.Consequently, the features of the obstruction are blurred and there is asignificant reduction in edge density. Accordingly, an image qualitymetric that reflects the edge density of an observed image is used todetect near-field obstructions, e.g., by comparing a result of themetric to a threshold value. Suitably, the threshold value may belearned from a training set of images with and without near-fieldobstructions.

As used herein, the terms edge generally refers to a location or pixelwhich exhibits a relatively high local gradient or value difference withrespect to a neighboring location or pixel or pixels such as istypically found at the edge of an in-focus object. The term edge as usedherein more generally includes any such features exhibiting theforegoing properties, e.g., such as a corner or the like. Additionally,the term edge detection as used herein more generally includes not onlythe detection of edges but also the detection of corners and/or otherlike features exhibiting the foregoing properties and/or the detectionof the foregoing properties themselves.

With reference now to FIG. 1, an automated and/or unattended camerasystem 10 includes a camera 12 for selectively capturing and/orobtaining an image of a scene within the camera's FoV. In practice, thecamera 12 may be a digital camera and may be either a still picturecamera or a video camera. When referring herein to a captured orotherwise obtained image from the camera 12, it is intended to mean animage from a picture camera or a still frame from a video camera.

As shown in FIG. 1, the camera 12 has an effective focal range FRbeginning at some distance D from the camera 12. Objects within the FRare generally in focus, while objects outside the FR are generally outof focus. In-focus objects or regions in an image captured by the camera12 will generally appear sharp and/or crisp (e.g., with well-definededges and/or high local gradients), while out-of-focus object or regionsin an image captured by the camera 12 will generally appear blurryand/or fuzzy (e.g., without well-defined edges and/or low localgradients). For example, objects and/or regions in the camera's FoV thatare sufficiently outside the FR and near the camera 12, e.g., in anear-field 20 of the camera's FoV, will generally be out-of-focus.Objects in the near-field 20 of the camera's FoV are nominally referredto herein as near-field obstructions. Typical near-field obstructionsinclude but are not limited to: plant growth and/or foliage; icebuild-up on the camera lens or porthole; graffiti or debris on thecamera lens or porthole; etc.

In the illustrated embodiment, the system 10 further includes a computer30 or the like that is remotely or otherwise in communication with thecamera 12. Suitably, the computer 30 obtains or otherwise receives andanalyzes images captured by the camera 12 in order to automaticallymonitor, detect and/or report near-field obstructions. In practice, theimage obtained or received and analyzed by the computer 30 is a digitalimage, e.g., captured by a digital camera. Optionally, the computer 30may receive an analog feed which is in turn digitized to obtain adigital image for analysis. In one suitable embodiment, the computer 30obtains or receives and analyzes essentially all the images captured bythe camera 12. Alternately, the computer 30 may obtain or receive andanalyze a representative sample or other subset of the images capturedby the camera 12 at periodic or intermittent intervals or otherwisechosen times. Suitably, the images may be transmitted from the camera 12to the computer 30 and/or analyzed in real time or near real time or inbatches or otherwise.

With reference now to FIG. 2, there is shown a flow chart illustratingan exemplary process 100 by which the obtained or captured images areanalyzed, e.g., by the computer 30. For purposes of the present example,reference is also made to FIG. 3 which shows an exemplary image 200captured by the camera 12 and that may be so analyzed. In particular,the image 200 includes a near-field obstruction 202, which in this caseis plant foliage. Notably, the near-field obstruction 202 appearsout-of-focus, e.g., as compared to the remainder of the image 200.

As shown in step 102, an image is obtained. For example, the image 200may be captured by the camera 12 and transmitted to the computer 30 foranalysis.

At step 104, edge or gradient detection is applied to the obtainedimage, e.g., to generate and edge or gradient map 300 such as the oneshown in FIG. 4. As shown, the light or white pixels represent edgepixels, i.e., pixels having a high gradient or value difference withrespect to neighboring pixels, and the dark or black pixels representnon-edge pixels, i.e., pixels having a low gradient or value differencewith respect to neighboring pixels. Suitably, a gradient-base edgedetector, e.g., such as a Sobel edge detector, is employed. Alternately,a soft-edge detector, e.g., such as a Canny edge detector may be used.However, the latter employs thresholding with hysteresis on theintensity gradient image while tracing edges to enforce edge continuity.Thus, it may include pixels as edge pixels that are relatively low inlocal gradient intensity. Regardless, it is notable in the gradient map300 that a relatively continuous dark area 302 (e.g., as compared to theremainder of the gradient map 300) corresponds to essentially the samearea as the near-field obstruction 202 in the image 200.

At step 106, local averaging or similar processing is applied to theedge or gradient map, e.g., to smooth out the spurious nature of edgepixels. Suitably, this is accomplished by defining a local window ofsuitable size, e.g., centered around a pixel (i,j) of the edge map. Inpractice, the window may then be moved or advance over successive pixelsin the edge map. Suitably, for a given pixel (i,j) around which thewindow is placed, a binary value is assigned thereto depending uponwhether or not an edge pixel resides within the window. For example, avalue of 1 may be assigned to a pixel (i,j) centered in the window, ifan edge pixel resides anywhere in the window, otherwise a value of zeromay be assigned to the pixel (i,j) centered in the window, if an edgepixel does not reside anywhere in the window. The window may then bemoved or advanced to be centered or otherwise placed around the nextpixel in the edge map which is in turn likewise assigned a binary value.In this manner, a binary image (i.e., a localized edge or gradientdensity map) may be generated where the pixel values represent orindicate a local edge content.

At step 108, connected component analysis is applied to the binary imageto identify clusters or connected or neighboring groups of pixelstherein corresponding to edge-free regions. That is to say, theconnected component analysis identifies cluster of pixels having likebinary values indicative of edge-free content (e.g., having a binaryvalue of zero in the present example). In this manner, region ofsignificant size in the image are found which lack high frequencycontent or large gradient changes.

At step 110, the edge content is suitably summarized to generate anoverall edge or gradient density metric. In practice, the edge contentmay be summarized by calculating or otherwise determining a scalar valuewhich measures or otherwise represents the amount of edge-free regionsidentified by the connected component analysis, e.g., as a percentage ofthe total image area.

At decision step 112, the metric result (e.g., the generated scalarvalue) is compared to a set or otherwise determined threshold orthreshold condition. If the former does not satisfy the thresholdcondition (e.g., if the metric result or scalar value is lower than thethreshold), then a near-field obstruction may be deemed to have beendetected and the process 100 may continue to step 114, otherwise if theformer does satisfy the threshold condition (e.g., if the metric resultor scalar value meets or exceeds the threshold), then a near-fieldobstruction may not be deemed to have been detected and the process 100may end.

In one suitable embodiment, an appropriate threshold may be learnedand/or set or determined based upon a statistical analysis of a set oftraining images obtained with and without near-field obstructions. Forexample, each training image may be subjected to the process 100 or asimilar process in order to generate an overall edge or gradient densitymetric therefor (such as the aforementioned scalar value). Accordingly,a nominal distribution of the metrics obtained from the training imagesmay be produced, and based on this distribution, a suitable thresholdmay be determined or selected.

At step 114, a suitable notification of the detected near-fieldobstruction is provided. For example, the computer 30 may provide such anotification by way of a visual indication, audible signal, display orsending of a suitable message, activation of a humanly perceivable alertor alarm, etc.

Under certain conditions, edge density may not always exhibit suitablehomogeneity in region of interest in an image scene. For example, sunglare and/or overexposure during daytime may reduce the overall or localedge density. Accordingly, certain conditions may optionally be detectedbefore applying the process 100 to an image. For example, an image maybe subjected to suitable algorithms, processes and/or analysis to detectsun glare and/or overexposure and/or other errors (e.g., which couldpotentially invalidate the results and/or otherwise interfere with theprocess 100), and if no such conditions are detected, then analysis inaccordance with the process 100 may be executed, otherwise if one ormore of such conditions are detected, then execution of the process 100may be forgone.

The above elements, components, processes, methods, apparatus and/orsystems have been described with respect to particular embodiments. Itis to be appreciated, however, that certain modifications and/oralteration are also contemplated.

It is to be appreciated that in connection with the particular exemplaryembodiment(s) presented herein certain structural and/or functionfeatures are described as being incorporated in defined elements and/orcomponents. However, it is contemplated that these features may, to thesame or similar benefit, also likewise be incorporated in other elementsand/or components where appropriate. It is also to be appreciated thatdifferent aspects of the exemplary embodiments may be selectivelyemployed as appropriate to achieve other alternate embodiments suitedfor desired applications, the other alternate embodiments therebyrealizing the respective advantages of the aspects incorporated therein.

It is also to be appreciated that any one or more of the particulartasks, steps, processes, analysis, methods, functions, elements and/orcomponents described herein may suitably be implemented via hardware,software, firmware or a combination thereof. For example, the computer30 may include a processor, e.g., embodied by a computing or otherelectronic data processing device, that is configured and/or otherwiseprovisioned to perform one or more of the tasks, steps, processes,analysis, methods and/or functions described herein. For example, thecomputer 30 or other electronic data processing device employed in thesystem 10 may be provided, supplied and/or programmed with a suitablelisting of code (e.g., such as source code, interpretive code, objectcode, directly executable code, and so forth) or other like instructionsor software or firmware (e.g., such as an application to perform and/oradminister the processing and/or image analysis described herein), suchthat when run and/or executed by the computer or other electronic dataprocessing device one or more of the tasks, steps, processes, analysis,methods and/or functions described herein are completed or otherwiseperformed. Suitably, the listing of code or other like instructions orsoftware or firmware is implemented as and/or recorded, stored,contained or included in and/or on a non-transitory computer and/ormachine readable storage medium or media so as to be providable toand/or executable by the computer or other electronic data processingdevice. For example, suitable storage mediums and/or media can includebut are not limited to: floppy disks, flexible disks, hard disks,magnetic tape, or any other magnetic storage medium or media, CD-ROM,DVD, optical disks, or any other optical medium or media, a RAM, a ROM,a PROM, an EPROM, a FLASH-EPROM, or other memory or chip or cartridge,or any other tangible medium or media from which a computer or machineor electronic data processing device can read and use. In essence, asused herein, non-transitory computer-readable and/or machine-readablemediums and/or media comprise all computer-readable and/ormachine-readable mediums and/or media except for a transitory,propagating signal.

Optionally, any one or more of the particular tasks, steps, processes,analysis, methods, functions, elements and/or components describedherein may be implemented on and/or embodiment in one or more generalpurpose computers, special purpose computer(s), a programmedmicroprocessor or microcontroller and peripheral integrated circuitelements, an ASIC or other integrated circuit, a digital signalprocessor, a hardwired electronic or logic circuit such as a discreteelement circuit, a programmable logic device such as a PLD, PLA, FPGA,Graphical card CPU (GPU), or PAL, or the like. In general, any device,capable of implementing a finite state machine that is in turn capableof implementing the respective tasks, steps, processes, analysis,methods and/or functions described herein can be used.

Additionally, it is to be appreciated that certain elements describedherein as incorporated together may under suitable circumstances bestand-alone elements or otherwise divided. Similarly, a plurality ofparticular functions described as being carried out by one particularelement may be carried out by a plurality of distinct elements actingindependently to carry out individual functions, or certain individualfunctions may be split-up and carried out by a plurality of distinctelements acting in concert. Alternately, some elements or componentsotherwise described and/or shown herein as distinct from one another maybe physically or functionally combined where appropriate.

In short, the present specification has been set forth with reference topreferred and/or other embodiments. Obviously, modifications andalterations will occur to others upon reading and understanding thepresent specification. It is intended that the invention be construed asincluding all such modifications and alterations insofar as they comewithin the scope of the appended claims or the equivalents thereof.

What is claimed is:
 1. A method for detecting an obstruction within afield of view of a camera from an image captured by the camera, saidmethod comprising: analyzing the image, said analyzing of the imagecomprising: applying edge detection to the image; identifying regions ofthe image lacking edge content; and, comparing a size of the identifiedregions to a threshold; and determining if there is an obstruction basedupon a result of said comparison.
 2. The method of claim 1, whereinapplying said edge detection generates an edge map including a pluralityof pixels, where each pixel may be either an edge pixel or not an edgepixel, and said analyzing further comprises: local smoothing of saidmap.
 3. The method of claim 2, wherein said local smoothing comprises:placing a window around a given pixel of the map; assigning a valueassociated with the given pixel based upon whether or not any pixelwithin the window is an edge pixel; and repeating said placing and saidassigning for successive pixels of the map.
 4. The method of claim 3,wherein said identifying comprises: performing connected componentanalysis on the smoothed map.
 5. The method of claim 1, wherein saidanalyzing further comprising: generating a scalar value which representsthe size of the identified regions.
 6. The method of claim 5, whereinthe scalar value is a percentage relative to an entire area of theimage.
 7. The method of claim 1, wherein said threshold is selectedbased on a statistical analysis of a set of training images including atleast one image obtained by a camera with an obstruction in its field ofview and at least one image obtained by a camera without an obstructionin its field of view.
 8. The method of claim 1, said method furthercomprising: providing notification of a detected obstruction.
 9. Anapparatus that executes the method of claim
 1. 10. A non-transitorymachine-readable medium including a computer program which when executedperforms the method of claim
 1. 11. A method for detecting anobstruction within a field of view of a camera from an image captured bythe camera, said method comprising: analyzing the image with a computerprocessor to identify regions of the image which are out of focus;comparing a size of the identified regions to a threshold; anddetermining if there is an obstruction based upon a result of saidcomparison.
 12. A camera system comprising: a camera that obtains animage; and an image processor that analyzes said image to determine ifthere is an obstruction in the camera's field of view, wherein saidanalyzing includes: applying edge detection to the image; identifyingregions of the image lacking edge content; comparing a size of theidentified regions to a threshold; and determining if there is anobstruction based upon a result of said comparison.
 13. The camerasystem of claim 12, wherein applying said edge detection generates anedge map including a plurality of pixels, where each pixel may be eitheran edge pixel or not an edge pixel, and said analyzing furthercomprises: local smoothing of said map.
 14. The camera system of claim13, wherein said local smoothing comprises: placing a window around agiven pixel of the map; assigning a value associated with the givenpixel based upon whether or not any pixel within the window is an edgepixel; and repeating said placing and said assigning for successivepixels of the map.
 15. The camera system of claim 14, wherein saididentifying comprises: performing connected component analysis on thesmoothed map.
 16. The camera system of claim 12, wherein said analyzingfurther comprising: generating a scalar value which represents the sizeof the identified regions.
 17. The camera system of claim 16, whereinthe scalar value is a percentage relative to an entire area of theimage.
 18. The camera system of claim 12, wherein said threshold isselected based on a statistical analysis of a set of training imagesincluding at least one image obtained by a camera with an obstruction inits field of view and at least one image obtained by a camera without anobstruction in its field of view.